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AT PRESENT, THE PART OF ANY TEACHER EFFECTIVENESS 
CRITERION THAT CAN BE PREDICTED WITH A SELECTION TEST IS 
PROBABLY IRRELEVANT TO TEACHER COMPETENCE. TESTING THE 
VALIDITY OF PREDICTORS OF TEACHER COMPETENCE IS IMPOSSIBLE 
BECAUSE IT WOULD REQUIRE HIRING A SIZABLE RANDOM SAMPLE OF 
ALL WHO APPLY FOR POSITIONS, WITHOUT PRIOR SCREENING. 

FURTHER, TEACHER APTITUDE TESTS WRONGLY ASSUME THAT THE 
FACTORS IN SUCCESSFUL TEACHING OPERATE PRIOR TO THE START OF 
TEACHING IN A SPECIFIC SCHOOL OR SCHOOL SYSTEM. INSTEAD, 
TEACHING BEHAVIOR PROBABLY VARIES WITH THE TEACHING 
SITUATION. ACCORDINGLY, AN ACHIEVEMENT RATHER THAN A 
PREDICTIVE MODEL SHOULD BE USED IN HIRING NEW TEACHERS. FAST 
LEARNING (E.G., AS MEASURED BY COLLEGE GRADES) IS ONE SUCH 
MEASURE. TEACHING PERFORMANCE IS ANOTHER ACHIEVEMENT MEASURE. 
THAT IS, IF, AFTER ONE YEAR, THE PROBATIONARY TEACHER HAS NOT 
LEARNED TO TEACH, HE SHOULD NOT BE REHIRED. THIS DCOUMENT 
APPEARED IN GILBERT, H.B. , AND LANG, G. , "TEACHER SELECTION 
METHODS" NEW YORK, 1967. (RD) 
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Some Notes on Validating Teacher Selection Pi’ocedures 

Donald M. Medley- 
Educational Testing Service 



As far as I know, all teacher selection procedures pres- 
ently used are based on the model of aptitude testing; in other 
words, in constructing selection instruments the effort has been 
to devise a battery which would predict success on the job. Val- 
idation studies have, accordingly, sought to establish predictive 
validity against some kind of a criterion measure of teacher ef- 
fectiveness obtained after the teacher has been admitted to employ 
ment. 



A selection battery that could do this job fairly well 
would be useful indeed; it would enable the selection agency to 
compile a list of candidates with the candidate who would make the 
best teacher at the top and the one who would make the poorest at 
the bottom. Then it would be possible to appoint as many teachers 
as were needed in a given year, beginning at the top of the list, 
knowing that the best possible set of candidates had been chosen. 
This is a beautiful ideal; but it just will not work in practice. 

To niy mind, the sooner all attempts to validate selection proce- 
dures in this way are abandoned the better. 

In the first place, when you consider the nature of what 
you are trying to predict— teacher competence— it seems highly im- 
probable that it can ever be measured with a paper-and-pencil test, 
or any other de-vice which could conceivably be used on the scale 
necessary for teacher selection in large cities. There is con- 
siderable experimental data which confirms this pessimistic point 
of view. Most of the predictive validities obtained in studies 
done in the past have been below .30; very few have exceeded .UO. 

And the improvement in predictive efficiency obtained with such 
small correlations is practically negligible. 

In the second place, even if tha correlations obtained 
were large enough to improve selection, their value would be sus- 
pect because of the limited validity of the criteria on which they 
would have to be based. At the present state of the art of measur- 
ing teacher competence, it is fair to say that the part of any 
teacher effectiveness criterion we can predict with a selection 
test is probably irrelevant to teacher competence anyhow. 

Finally, it is extremely difficult actually to carry out 
a validity study based on this model, since to do so requires that— 



for expsrijnsntal purpos6s--a good sized random sample of all who^ 
apply for positions in a school system be admitted to teaching with- 
out any kind of selection. This is probably illegal, but in any 
case is not practical, in most large school systems. 

A more fundamental reason for abandoning the aptitude test 
model is the fact that its use is based on an untenable assumptionj 
the assumption that che major factors which determine whether a can- 
didate will succeed or fail as a teacher operate before he enters 
the school system. Such things as what kind of a school and neigh- 
borhood the teacher is assigned to, the characteristics of his 
pxiplls and the facilities and materials available to him when he 
tries to teach them, and the araoimt and kind of support he receives 
from his peers and superiors in the school system are seen as dis- 
tinctly less important than such things as what college courses he 
has had and what he learned in them, whether he has worked in sum- 
mer camps during his undergraduate days, and how happy his childhood 
was, in determining his future as a teacher. This assumption is 
clearly implied by a conception of the selection problem as one of 
identifying among applicants for positions those predestined to be- 
come competent teachers . 

It seems more realistic not to assume that the future of 
any of the candidates has been (or should have been) decided at the 
time when the selection takes place, but only that the candidates 
will vary in the degree to which they have mastered that part of 
their preparation which can be obtained before they enter the sys- 
tem. The selection problem would then be seen as one of assessing 
past learning rather than one of predicting future performance. 

The problem of validating a selection battery would then 
become a matter of content validity rather than of predictive yal- 
idityj and this kind of validity is much more likely to be achieved 
by a paper -and -pencil test or one of the other techniq'''.es which a 
practical selection battery is likely to contain. From this point 
of view it may be said that teacher selection should be based on an 
achievement model rather than a prediction model. 

After a teacher has been admitted to probationary status 
in the school system, we are still faced with a problem of eliminat- 
ing those candidates who have satisfactorily completed their pre- 
service training but cannot teach successfully. This could be re- 
garded as a problem in prediction^ but I prefer to say that it is 
another problem in achievement testing. The first years of a 
teacher’s career should be viewed as a part of his training^ if by 
the end of his probationary period he has not learned to teach, he 
is not ready to be admitted to permanent tenure and should probably 
be let go. 
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the teaching of a standard lesson under observation, or an 
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could be used at the pre-service level. 

If we adopt the achievement model for validating selection 

improving the quality of teaching in the schools. 

There is one approach to this problem which (so far as I 
know) has never been tried out. This approach would 
tinuTOS and routine monitoring of the quality of teachers in 

ductSn"Snrw^eTa^''here?“^iJ^^ T smP}e 
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obtain a quite satisfactory estimate of the average quail y 
entire output by inspecting only a sample of it. 

In similar fashion, the city school system f 

stratified random sample of all the teachers 

system each year, and make a thorough ®^/^®Sts1f thf study as 

TelttZ tt^Sy tneSndtSl wouid- be kept confidential, 

Teef mde tSt they would not affect his ^-^^.^'tShtr ’ ptS- 

Stion'^SVthole'?® Sth dtta wtuld ^ttide precise information about 
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on the basis of which changes could be made. 

The fact that only group scores »ould be used 
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Sfety SpSeSnftiScSsuting procedures. Although we do not 
tively expensive d measurine the competence of indi- 

presently have the capability oi m a ^ pnnfident that we 

vidual teachers reliably and economically^ I ^ teachers 

Ituld reliably estimate the average, competence of all the teache.s 
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in a system, using techniques already available to us, at a per 
teacher cost which would be quite reasonable. 

I have introduced this idea of quality control of teach- 
ers in a system in the context of the problem of monitoring teach- 
er selection policies and procedures, because that is our immediate 
concern herej but I would point out the obvious fact that such in- 
formation would have many other uses, some of which might be re- 
garded as more important than this one. I see no reason why all of 
these purposes should not be achieved at the same time. Nor do I 
see any other feasible way of achieving the one we are concerned 
with— that of assessing the effects of selection policies and 
practices as they are used. 
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